In this document, we intend to investigate the following key questions, assuming a fixed array of \(10^6\) SNPs and a quantitative trait in which SNP effect sizes follow a normal distribution:

1. When is Winner’s Curse a problem?

In this section, we look at the average number of significant SNPs, the average proportion of these significant SNPs that have association estimates more extreme than their true effect size and the average MSE of significant SNPs at two different thresholds; the common genome-wide significance threshold of \(5 \times 10^{-8}\) and a higher threshold of \(5 \times 10^{-4}\). We consider these properties under certain combinations of values for the following parameters:

  1. sample size - n_samples
  2. heritability - h2
  3. polygenicity, i.e. proportion of effect SNPs - prop_effect
  4. selection coefficient - S

The 24 different combinations that we will investigate throughout this document are detailed below:

Scenario n_samples h2 prop_effect S
1 30,000 0.3 0.010 -1
2 300,000 0.3 0.010 -1
3 30,000 0.8 0.010 -1
4 300,000 0.8 0.010 -1
5 30,000 0.3 0.001 -1
6 300,000 0.3 0.001 -1
7 30,000 0.8 0.001 -1
8 300,000 0.8 0.001 -1
9 30,000 0.3 0.010 0
10 300,000 0.3 0.010 0
11 30,000 0.8 0.010 0
12 300,000 0.8 0.010 0
Scenario n_samples h2 prop_effect S
13 30,000 0.3 0.001 0
14 300,000 0.3 0.001 0
15 30,000 0.8 0.001 0
16 300,000 0.8 0.001 0
17 30,000 0.3 0.010 1
18 300,000 0.3 0.010 1
19 30,000 0.8 0.010 1
20 300,000 0.8 0.010 1
21 30,000 0.3 0.001 1
22 300,000 0.3 0.001 1
23 30,000 0.8 0.001 1
24 300,000 0.8 0.001 1

\(~\) \(~\) \(~\)

Running the code provided in nsig_prop_bias_100sim.R, we obtain the following results. Note that prop_x refers to the proportion of significant SNPs which have been found to be significantly overestimated, i.e. those SNPs in which \(\left| \hat\beta_{i} \right| > \left| \beta_{i} \right| + 1.96\cdot\sigma_{i}\).

Scenario n_samples h2 prop_effect S n_sig 5e-8 prop_bias 5e-8 prop_x 5e-8 mse 5e-8 n_sig 5e-4 prop_bias 5e-4 prop_x 5e-4 mse 5e-4 sd(n_sig) 5e-8 sd(prop_bias) 5e-8 sd(prop_x) 5e-8 sd(mse) 5e-8 sd(n_sig) 5e-4 sd(prop_bias) 5e-4 sd(prop_x) 5e-4 sd(mse) 5e-4
1 30,000 0.3 0.010 -1 0.70 1.0000 0.9000 0.001573 610.99 0.9996 0.9165 0.001919 0.745 0.0000 0.2950 0.001234 23.070 0.0008 0.0108 0.000105
2 300,000 0.3 0.010 -1 848.63 0.7619 0.0895 0.000022 3201.35 0.7461 0.2083 0.000049 18.145 0.0142 0.0103 0.000002 47.967 0.0070 0.0071 0.000003
3 30,000 0.8 0.010 -1 31.85 0.9804 0.4020 0.000598 1089.98 0.9597 0.5647 0.001194 5.208 0.0280 0.0834 0.000198 35.158 0.0053 0.0136 0.000060
4 300,000 0.8 0.010 -1 2760.68 0.6284 0.0484 0.000017 5349.44 0.6393 0.1305 0.000035 30.091 0.0084 0.0038 0.000001 43.729 0.0069 0.0045 0.000002
5 30,000 0.3 0.001 -1 86.90 0.7591 0.0892 0.000214 772.12 0.8957 0.6724 0.001474 6.317 0.0437 0.0290 0.000054 25.913 0.0090 0.0140 0.000088
6 300,000 0.3 0.001 -1 568.70 0.5509 0.0330 0.000016 1211.96 0.7303 0.4300 0.000099 14.074 0.0225 0.0078 0.000002 24.787 0.0128 0.0127 0.000006
7 30,000 0.8 0.001 -1 276.26 0.6273 0.0464 0.000168 986.59 0.8043 0.5276 0.001198 10.413 0.0271 0.0123 0.000027 26.682 0.0117 0.0138 0.000075
8 300,000 0.8 0.001 -1 727.10 0.5257 0.0287 0.000016 1326.17 0.7068 0.3971 0.000093 12.491 0.0171 0.0061 0.000001 24.072 0.0127 0.0105 0.000005
9 30,000 0.3 0.010 0 1.45 1.0000 0.8610 0.001661 627.16 0.9985 0.8888 0.001825 1.298 0.0000 0.2745 0.005559 25.180 0.0015 0.0128 0.000090
10 300,000 0.3 0.010 0 882.45 0.7297 0.0807 0.000012 3054.78 0.7427 0.2176 0.000046 18.435 0.0152 0.0098 0.000001 45.422 0.0070 0.0066 0.000002
11 30,000 0.8 0.010 0 48.06 0.9509 0.2908 0.000245 1110.51 0.9407 0.5435 0.001091 6.350 0.0301 0.0645 0.000050 29.205 0.0068 0.0123 0.000060
12 300,000 0.8 0.010 0 2586.78 0.6204 0.0477 0.000011 5032.06 0.6455 0.1375 0.000032 32.771 0.0096 0.0038 0.000000 45.218 0.0062 0.0046 0.000002
13 30,000 0.3 0.001 0 88.32 0.7254 0.0774 0.000116 755.09 0.8953 0.6820 0.001491 6.377 0.0480 0.0279 0.000025 22.799 0.0097 0.0131 0.000077
14 300,000 0.3 0.001 0 531.27 0.5560 0.0334 0.000012 1183.18 0.7402 0.4407 0.000100 12.431 0.0215 0.0074 0.000001 27.843 0.0121 0.0091 0.000006
15 30,000 0.8 0.001 0 257.44 0.6234 0.0473 0.000106 953.63 0.8130 0.5449 0.001202 8.936 0.0276 0.0120 0.000013 23.262 0.0100 0.0115 0.000070
16 300,000 0.8 0.001 0 691.46 0.5268 0.0301 0.000013 1301.33 0.7114 0.4040 0.000092 13.526 0.0185 0.0066 0.000001 29.469 0.0110 0.0123 0.000005
17 30,000 0.3 0.010 1 2.55 0.9941 0.7339 0.000610 641.18 0.9966 0.8643 0.001777 1.623 0.0405 0.3034 0.000674 25.497 0.0022 0.0126 0.000107
18 300,000 0.3 0.010 1 919.28 0.7031 0.0706 0.000010 2902.71 0.7313 0.2228 0.000046 15.799 0.0142 0.0079 0.000001 38.290 0.0086 0.0073 0.000003
19 30,000 0.8 0.010 1 68.13 0.9178 0.2410 0.000198 1141.84 0.9208 0.5196 0.001047 7.795 0.0370 0.0486 0.000087 29.135 0.0077 0.0144 0.000063
20 300,000 0.8 0.010 1 2433.27 0.6105 0.0462 0.000009 4634.50 0.6464 0.1461 0.000032 30.705 0.0107 0.0047 0.000000 48.828 0.0078 0.0046 0.000002
21 30,000 0.3 0.001 1 93.15 0.7056 0.0684 0.000096 742.12 0.8944 0.6949 0.001509 5.960 0.0428 0.0273 0.000015 23.899 0.0102 0.0140 0.000080
22 300,000 0.3 0.001 1 482.94 0.5516 0.0329 0.000009 1124.74 0.7510 0.4635 0.000103 12.742 0.0244 0.0072 0.000001 24.808 0.0116 0.0117 0.000006
23 30,000 0.8 0.001 1 244.29 0.6120 0.0448 0.000089 917.81 0.8199 0.5673 0.001249 9.237 0.0272 0.0142 0.000010 25.125 0.0110 0.0137 0.000084
24 300,000 0.8 0.001 1 632.74 0.5336 0.0294 0.000010 1235.79 0.7201 0.4192 0.000094 14.730 0.0194 0.0071 0.000001 25.984 0.0121 0.0112 0.000005

\(~\) \(~\) \(~\)

★ It is important to note here that for scenarios 1, 9 and 17, very few significant SNPs are detected on average. In some instances, we may even find that no SNPs are deemed significant at a threshold of \(5 \times 10^{-8}\). We must keep this observation in mind going forward as we investigate the performance of methods under these three scenarios.

For both thresholds, the average number of significant SNPs increases as sample size increases, as expected. It also increases with heritability. However, the effect of changing prop_effect is more interesting. Decreasing the proportion of effect SNPs from 0.01 to 0.001 results in the number of significant SNPs increasing for a sample size of 30,000 while we witness the number of SNPs passing the genome-wide significance threshold decreasing for a larger sample size of 300,000.

Furthermore, increasing sample size and increasing heritability from 0.3 to 0.8 all tend to decrease the fraction of significant SNPs whose estimates are more extreme than their true effect size. Decreasing polygenicity from 0.01 to 0.001 also has this same effect at a significance threshold of \(5 \times 10^{-8}\).

As sample size increases from 30,000 to 300,000 in all scenarios, prop_x, the proportion of significant SNPs which have been found to be significantly overestimated, decreases. This is a strong indicator that as the value of n_samples increases, we expect the bias induced by Winner’s Curse to be less of a problem among significant SNPs. However, this could also be due to the fact that as sample size increases, the number of significant SNPs passing the genome-wide significance threshold of 5e-8 also increases.

In order to gain a better insight into the information detailed in the above table, we simulate a single set of GWAS summary statistics and plot \(z\) vs \(\text{bias}\) in which \(\text{bias} = \hat\beta - \beta\) for each of the 24 different scenarios. On all figures, the bright red line corresponds to the significance threshold of \(5 \times 10^{-8}\) while the darker red line relates to \(5 \times 10^{-4}\). The points corresponding to SNPs which are significantly overestimated and are significant at a threshold of \(5 \times 10^{-4}\) are coloured in navy .

2. Evaluating methods using a significance threshold of \(5 \times 10^{-8}\)

Using the code detailed in norm_5e-8_20sim.R and a total of 20 simulations, we evaluated six different Winner’s Curse methods across each of the 24 scenarios using the following three bias evaluation metrics:

  1. The average fraction of significant SNPs in which their association estimates have been improved due to method implementation - flb
  2. The average change in average MSE of significant SNPs due to method implementation - mse
  3. The average relative change in average MSE of significant SNPs due to method implementation - rel_mse

Note: All averages are obtained over only those simulations in which at least one significant SNP was detected.

Firstly, the fraction of \(n\) significant SNPs in which their association estimates have been improved due to method implementation may be mathematically described as: \[\frac{1}{n} \; \sum_{i=1}^{n}\mathbb{I} \left\{ \left| \hat\beta_i - \beta_i \right| > \left|\hat\beta_{\text{adj,}i} - \beta_i\right| \right\},\]in which \(\left| \frac{\hat\beta_i}{\hat\sigma_i} \right| > Z_{\frac{\alpha}{2}}\) for all \(i = 1,...,n\), where \(\hat\beta_i\) is the estimated naive effect size of SNP \(i\), \(\beta_i\) is its true effect size and \(\hat\beta_{\text{adj,}i}\) is its new effect size estimate obtained as a result of application of the Winner’s Curse adjustment method of interest. The significance threshold is represented by \(\alpha\).

Using the same notation, the average MSE over \(n\) significant SNPs is defined as: \[\frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2.\] Thus, using the above, we may formally define the change in average MSE of significant SNPs as: \[\frac{1}{n} \sum^n_{i=1} (\hat\beta_{\text{adj,}i} - \beta_i)^2 - \frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2\] and the relative change in average MSE of significant SNPs as: \[\frac{\frac{1}{n} \sum^n_{i=1} (\hat\beta_{\text{adj,}i} - \beta_i)^2 - \frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2}{\frac{1}{n} \sum^n_{i=1} (\hat\beta_i - \beta_i)^2}.\]

\(~\)

Results of the simulations are plotted. Error bars are also included in the plots. These figures allow us to see more clearly the scenarios in which it would be beneficial to apply a Winner’s Curse correction method and also, provide us with a better indication of which method we should use.

<<<<<<< HEAD

A replication method which selects significant SNPs from a discovery GWAS and then uses a replication GWAS of the same size to obtain association estimates for these SNPs is also included in the plots. This can be viewed as acting as a form of benchmark for the other methods.

Summary of results for flb contained in norm_5e-8_20sim.csv:

=======

Summary of results for flb contained in norm_5e-8_20sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 0.9000 0.9000 0.8500 0.7500 1.0000 0.9000 1.0000
2 300,000 0.3 0.010 -1 <<<<<<< HEAD 0.6396 0.4446 0.6034 0.5289 0.4974 0.5095 0.5612 ======= 0.6417 0.4368 0.6028 0.5320 0.5017 0.5134 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD 0.9140 0.7580 0.7802 0.6152 0.7338 0.6657 0.8166 ======= 0.8891 0.7522 0.7579 0.5865 0.6850 0.6306 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD 0.5673 0.2811 0.5319 0.5174 0.4865 0.5014 0.5138 ======= 0.5654 0.2814 0.5327 0.5148 0.4845 0.4990 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD 0.6312 0.3257 0.4595 0.5253 0.4911 0.5068 0.5631 ======= 0.6353 0.3602 0.4889 0.5328 0.5018 0.5134 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD 0.4926 0.1340 0.2759 0.5097 0.4778 0.5017 0.5002 ======= 0.4805 0.1341 0.2665 0.4993 0.4725 0.4932 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD 0.5230 0.2248 0.2974 0.5213 0.4886 0.5037 0.5118 ======= 0.5245 0.2213 0.3009 0.5152 0.4841 0.4963 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 <<<<<<< HEAD 0.4757 0.1053 0.3009 0.5028 0.4711 0.4894 0.5082 ======= 0.4788 0.0991 0.3036 0.5034 0.4772 0.4906 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD 0.9062 ======= 0.9375 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.9375 <<<<<<< HEAD 0.8438 0.9062 ======= 0.8750 0.7500 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.9375 <<<<<<< HEAD 0.9062 0.9375 ======= 0.8125 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 <<<<<<< HEAD 0.6212 0.4058 0.5772 0.5291 0.4977 0.5120 0.5466 ======= 0.6214 0.4054 0.5806 0.5349 0.5017 0.5153 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD 0.8435 0.6979 0.7417 0.5920 0.6757 0.6251 0.7557 ======= 0.8396 0.6581 0.7120 0.5782 0.6356 0.6007 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD 0.5589 0.2708 0.5246 0.5120 0.4817 0.4958 0.5092 ======= 0.5552 0.2711 0.5216 0.5117 0.4817 0.4968 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD 0.6006 0.3048 0.4487 0.5061 0.4820 0.4932 0.5441 ======= 0.5918 0.3020 0.4396 0.5192 0.4835 0.4918 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD 0.4862 0.1390 0.2821 0.5043 0.4742 0.4964 0.4942 ======= 0.4869 0.1382 0.2834 0.5100 0.4767 0.5010 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD 0.5159 0.2013 0.2860 0.5130 0.4872 0.5008 0.5100 ======= 0.5150 0.2139 0.2879 0.5155 0.4865 0.4987 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 <<<<<<< HEAD 0.4635 0.1089 0.3005 ======= 0.4587 0.1091 0.3000 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.5023 <<<<<<< HEAD 0.4653 ======= 0.4582 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.4833 0.5002
17 30,000 0.3 0.010 1 <<<<<<< HEAD 0.7792 0.8292 0.8083 0.6670 0.8708 0.7620 0.9358 ======= 0.8546 0.8685 0.8296 0.6102 0.8898 0.8370 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 <<<<<<< HEAD 0.5961 0.3647 0.5290 0.5219 0.4902 0.5033 0.5399 ======= 0.5970 0.3661 0.5346 0.5239 0.4910 0.5029 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD 0.7829 0.5712 0.6422 0.5588 0.5879 0.5679 0.6950 ======= 0.7977 0.6161 0.6807 0.5506 0.5915 0.5588 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 <<<<<<< HEAD 0.5510 0.2573 0.5109 0.5095 0.4803 0.4957 0.5121 ======= 0.5541 0.2583 0.5164 0.5134 0.4855 0.4988 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD 0.5875 0.2870 0.4261 0.5109 0.4843 0.4958 0.5459 ======= 0.5839 0.2741 0.4091 0.5132 0.4809 0.4945 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD 0.4799 0.1429 0.2779 0.5050 0.4704 0.4960 0.4955 ======= 0.4760 0.1508 0.2848 0.5102 0.4808 0.5015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD 0.4990 0.1928 0.2653 0.5044 0.4802 0.4915 0.5176 ======= 0.5052 0.1824 0.2652 0.5100 0.4827 0.4982 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD 0.4653 0.1098 0.3093 0.5043 0.4576 0.4790 0.5022 ======= 0.4579 0.1148 0.3095 0.5037 0.4628 0.4786 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

Fraction of significant SNPs with improved association estimates due to method implementation, using a significance threshold of \(5 \times 10^{-8}\):

<<<<<<< HEAD

\(~\) \(~\) \(~\) \(~\)

Summary of results for mse contained in norm_5e-8_20sim.csv:

=======

\(~\) \(~\) \(~\) \(~\)

Summary of results for mse contained in norm_5e-8_20sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 <<<<<<< HEAD -0.001739 -0.001747 -0.001756 -0.001440 -0.001631 -0.001592 -0.001604 ======= -0.001036 -0.001113 -0.001070 -0.000810 -0.001042 -0.000971 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 -0.000006 0.000001 -0.000004 0.000052 0.000023 <<<<<<< HEAD 0.000035 -0.000005 ======= 0.000035 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD -0.000435 -0.000453 -0.000464 0.000158 -0.000327 -0.000139 -0.000442 ======= -0.000397 -0.000382 -0.000399 0.000340 -0.000251 -0.000016 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 -0.000002 0.000005 0.000000 0.000035 0.000020 <<<<<<< HEAD 0.000026 -0.000001 ======= 0.000025 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD -0.000032 0.000130 0.000073 0.000599 0.000245 0.000391 -0.000040 ======= -0.000052 0.000128 0.000056 0.000553 0.000246 0.000371 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 0.000001 0.000007 0.000013 0.000014 0.000009 <<<<<<< HEAD 0.000012 0.000000 ======= 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 -0.000001 <<<<<<< HEAD 0.000134 0.000163 0.000322 0.000192 0.000243 -0.000015 ======= 0.000118 0.000139 0.000356 0.000198 0.000262 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 0.000001 0.000003 0.000007 0.000008 <<<<<<< HEAD 0.000023 0.000011 0.000000 ======= 0.000022 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD -0.000742 -0.000767 -0.000719 -0.000592 -0.000750 -0.000703 -0.000670 ======= -0.000482 -0.000506 -0.000459 -0.000173 -0.000431 -0.000342 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 -0.000003 0.000001 -0.000002 0.000026 0.000011 0.000017 <<<<<<< HEAD -0.000003 ======= >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD -0.000144 -0.000137 -0.000149 0.000184 -0.000065 0.000035 -0.000153 ======= -0.000153 -0.000135 -0.000146 0.000206 -0.000055 0.000050 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 -0.000001 0.000002 0.000000 0.000021 0.000012 0.000015 -0.000001
13 30,000 0.3 0.001 0 -0.000021 <<<<<<< HEAD 0.000073 0.000038 0.000292 0.000124 0.000193 -0.000028 ======= 0.000073 0.000046 0.000286 0.000129 0.000193 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 0.000000 0.000005 0.000009 0.000014 0.000009 0.000011 <<<<<<< HEAD 0.000000 ======= >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 -0.000003 <<<<<<< HEAD 0.000069 0.000105 0.000232 0.000121 0.000166 -0.000012 ======= 0.000064 0.000097 0.000205 0.000108 0.000148 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 0.000001 0.000004 0.000006 <<<<<<< HEAD 0.000011 0.000026 0.000014 0.000000 ======= 0.000013 0.000030 0.000015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD -0.000500 -0.000565 -0.000532 -0.000297 -0.000526 -0.000449 -0.000609 ======= -0.000416 -0.000426 -0.000399 -0.000083 -0.000398 -0.000282 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 -0.000002 0.000002 0.000000 0.000023 0.000011 0.000015 -0.000002
19 30,000 0.8 0.010 1 <<<<<<< HEAD -0.000113 -0.000083 -0.000102 0.000214 -0.000011 ======= -0.000112 -0.000085 -0.000101 0.000236 -0.000002 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.000095 -0.000125
20 300,000 0.8 0.010 1 -0.000001 0.000002 0.000000 0.000016 0.000009 0.000012 -0.000001
21 30,000 0.3 0.001 1 <<<<<<< HEAD -0.000015 0.000072 0.000046 0.000246 0.000109 0.000165 -0.000026 ======= -0.000010 0.000070 0.000045 0.000233 0.000107 0.000158 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 0.000000 0.000003 0.000007 0.000010 0.000007 0.000008 <<<<<<< HEAD 0.000000 ======= >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD 0.000003 ======= 0.000001 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.000056 <<<<<<< HEAD 0.000103 0.000156 0.000089 0.000116 -0.000005 ======= 0.000099 0.000160 0.000088 0.000116 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 0.000001 0.000002 0.000005 0.000008 <<<<<<< HEAD 0.000044 0.000016 0.000000 ======= 0.000037 0.000014 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

Change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\):

<<<<<<< HEAD

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in norm_5e-8_20sim.csv:

=======

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in norm_5e-8_20sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 <<<<<<< HEAD -0.8659 -0.8976 -0.8466 -0.6782 -0.8740 -0.8233 -0.8254 ======= -0.6604 -0.7586 -0.5970 -0.0416 -0.7533 -0.4873 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 <<<<<<< HEAD -0.3143 0.0630 -0.2040 2.5177 1.0966 1.6754 -0.2480 ======= -0.2989 0.0632 -0.1870 2.5004 1.0869 1.6622 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD -0.7314 -0.7086 -0.7393 0.3578 -0.5004 -0.1665 -0.7096 ======= -0.7280 -0.6790 -0.7147 0.6379 -0.4443 -0.0140 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD -0.0977 0.2689 -0.0143 2.0181 1.1567 1.5003 -0.0817 ======= -0.0898 0.2845 -0.0033 2.0018 1.1615 1.4939 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD -0.1375 0.7887 0.4699 3.3286 1.4118 2.2051 -0.1428 ======= -0.1757 0.7489 0.3820 2.7333 1.2761 1.8716 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD 0.0349 0.4142 0.8520 0.9163 0.6300 0.7364 -0.0251 ======= 0.0517 0.4211 0.8243 0.8484 0.5965 0.6886 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD -0.0032 0.7892 0.9577 1.8979 1.1281 1.4273 -0.0683 ======= -0.0035 0.7616 0.8963 2.2455 1.2639 1.6585 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 <<<<<<< HEAD 0.0638 0.2201 0.4378 0.5427 1.4651 0.6953 0.0091 ======= 0.0512 0.2099 0.4236 0.5236 1.4262 0.6775 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD -0.5376 -0.6701 -0.4098 0.1175 -0.6814 -0.3631 -0.6809 ======= -0.2439 -0.4177 -0.1850 0.7787 -0.4338 0.0504 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 <<<<<<< HEAD -0.2770 0.0366 -0.1718 2.2535 0.9473 1.4787 -0.2795 ======= -0.2636 0.0651 -0.1594 2.3332 1.0020 1.5453 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD -0.6387 -0.5912 -0.6478 0.8678 -0.2670 0.1896 -0.6672 ======= -0.6696 -0.5757 -0.6260 0.9705 -0.2137 0.2651 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD -0.1012 0.1955 -0.0384 1.9523 1.1193 1.4323 -0.1073 ======= -0.0982 0.2001 -0.0355 1.9867 1.0702 1.4377 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD -0.1800 0.6662 0.3534 2.5659 1.1128 1.7091 -0.2209 ======= -0.1805 0.7452 0.4794 2.6822 1.2278 1.8205 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD 0.0303 0.4008 0.7867 1.2410 0.7627 0.9444 -0.0137 ======= 0.0330 0.4196 0.7916 1.2425 0.7841 0.9560 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD -0.0099 0.6751 1.0223 2.2359 1.1727 1.6048 -0.1075 ======= -0.0224 0.6125 0.9225 1.9473 1.0361 1.4060 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 <<<<<<< HEAD 0.1443 0.3015 0.4888 0.8795 2.0278 1.0410 -0.0314 ======= 0.1168 0.3167 0.5312 1.0189 2.4554 1.2144 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD -0.6014 -0.6888 -0.6122 0.1929 -0.6644 -0.3321 -0.8068 ======= -0.7281 -0.7308 -0.6273 0.3158 -0.7049 -0.3138 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 <<<<<<< HEAD -0.2023 0.2016 -0.0206 2.2877 1.1015 1.5804 -0.2169 ======= -0.1962 0.1977 -0.0266 2.3515 1.1098 1.6144 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD -0.5828 -0.3981 -0.5037 1.2294 0.0019 0.4995 -0.6136 ======= -0.5992 -0.4470 -0.5346 1.2799 -0.0018 0.5187 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 <<<<<<< HEAD -0.0734 0.2387 0.0130 1.8655 1.0276 1.3627 -0.0955 ======= -0.0799 0.2242 0.0054 1.8330 1.0099 1.3388 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD -0.1503 0.7625 0.4876 2.5461 1.1361 1.7120 -0.2582 ======= -0.0957 0.7492 0.4913 2.4301 1.1282 1.6581 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD 0.0506 0.4088 0.8656 1.1302 0.7106 0.8616 -0.0083 ======= 0.0490 0.3649 0.8044 1.1342 0.7166 0.8661 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD 0.0373 0.6623 1.2072 1.8365 1.0397 1.3586 -0.0476 ======= 0.0113 0.6257 1.1099 1.7927 0.9795 1.3027 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD 0.1362 0.2605 0.4774 0.8255 4.3116 1.5529 -0.0373 ======= 0.1577 0.2511 0.4881 0.8140 3.6795 1.4035 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##         EB       FIQT         BR        cl1        cl2        cl3        rep 
## -0.2086333  0.1416583  0.1754542  1.4553375  0.8488625  0.9951000 -0.2618500

The methods are ranked according to the results above in ascending order:

##   EB FIQT   BR  cl1  cl2  cl3  rep 
##    2    3    4    7    5    6    1

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\):

<<<<<<< HEAD

3. Observations and discussion of above simulations

It is worth noting from the above simulations how the Winner’s Curse methods tend to break down, or equivalently, no longer make improvements based on the third evaluation metric, rel_mse when the proportion of effect SNPs is 0.001. This is a measure of polygenicity. That said, the empirical Bayes method performs extremely similar to just taking a replication estimate when the replication sample is of the same size as the discovery GWAS, i.e. n_samples is equivalent in both data sets and as we will see later, the empirical Bayes will outperform the replication method when the replication GWAS sample size becomes smaller than that of the discovery, at this threshold of 5e-8.

On all occasions, the conditional likelihood methods are performing poorly compared to the other methods.

=======

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

4. Evaluating methods using a significance threshold of \(5 \times 10^{-4}\)

Similar to part 2 above, we use the code detailed in norm_5e-4_10sim.R with a total of 10 simulations in order to evaluate seven different Winner’s Curse methods across each of the 24 scenarios. In the following investigations, we will concentrate on the final bias evaluation metric, rel_mse.

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in norm_5e-4_10sim.csv:

Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 <<<<<<< HEAD -0.9516 -0.9538 -0.8959 -0.8370 -0.7039 -0.7932 -0.9216 ======= 0.9428 0.9398 0.9738 0.9679 0.9917 0.9817 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 <<<<<<< HEAD -0.4013 -0.2231 -0.2830 -0.2135 -0.3549 -0.3199 -0.6819 ======= 0.5933 0.4565 0.5157 0.6042 0.6072 0.6037 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD -0.7634 -0.7475 -0.7374 -0.6905 -0.6600 -0.7033 -0.8664 ======= 0.7600 0.7123 0.7525 0.7959 0.8491 0.8233 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD -0.2859 -0.0602 -0.1729 -0.1374 -0.2304 -0.2131 -0.5530 ======= 0.5477 0.3488 0.4929 0.5574 0.5481 0.5541 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD -0.8184 -0.7769 -0.7278 -0.7638 -0.6496 -0.7286 -0.8955 ======= 0.7826 0.7264 0.7520 0.8334 0.8364 0.8352 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD -0.7582 -0.6891 -0.5591 -0.7297 -0.6083 -0.6904 -0.8329 ======= 0.6689 0.5026 0.5693 0.7127 0.6950 0.7110 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD -0.7888 -0.7142 -0.6462 -0.7525 -0.6325 -0.7150 -0.8723 ======= 0.7009 0.6089 0.6316 0.7641 0.7575 0.7607 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 <<<<<<< HEAD -0.7390 -0.7006 -0.5663 -0.7322 -0.4960 -0.6632 -0.8239 ======= 0.6451 0.4477 0.5598 0.6945 0.6699 0.6873 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD -0.9602 -0.9623 -0.8920 -0.8364 -0.6981 -0.7889 -0.9228 ======= 0.9220 0.9200 0.9537 0.9547 0.9807 0.9701 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 <<<<<<< HEAD -0.5172 -0.4540 -0.4662 -0.4573 -0.5001 -0.5093 -0.7680 ======= 0.5986 0.4589 0.5221 0.6112 0.6140 0.6121 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD -0.8003 -0.8057 -0.7681 -0.7498 -0.6698 -0.7350 -0.8929 ======= 0.7378 0.6852 0.7219 0.7805 0.8228 0.7987 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD -0.3762 -0.2532 -0.3139 -0.2712 -0.3518 -0.3419 -0.6549 ======= 0.5486 0.3555 0.4992 0.5635 0.5543 0.5604 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD -0.8694 -0.8520 -0.7850 -0.8045 -0.6715 -0.7598 -0.9124 ======= 0.7926 0.7365 0.7639 0.8397 0.8444 0.8413 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD -0.7661 -0.7077 -0.5836 -0.7417 -0.6210 -0.7032 -0.8567 ======= 0.6651 0.5095 0.5754 0.7199 0.7069 0.7174 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD -0.8080 -0.7669 -0.6897 -0.7596 -0.6432 -0.7231 -0.8884 ======= 0.7153 0.6211 0.6414 0.7731 0.7680 0.7719 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 <<<<<<< HEAD -0.7355 -0.7030 -0.5716 -0.7424 -0.4546 -0.6598 -0.8420 ======= 0.6454 0.4573 0.5743 0.6967 0.6748 0.6879 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD -0.9483 -0.9552 -0.8843 -0.8346 -0.6971 -0.7874 -0.9221 ======= 0.9000 0.8965 0.9307 0.9439 0.9699 0.9588 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 <<<<<<< HEAD -0.5410 -0.4831 -0.4843 -0.5187 -0.5206 -0.5474 -0.7940 ======= 0.5996 0.4591 0.5278 0.6153 0.6189 0.6155 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD -0.7916 -0.7933 -0.7542 -0.7493 -0.6628 -0.7304 -0.8932 ======= 0.7353 0.6724 0.7056 0.7751 0.8103 0.7917 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 <<<<<<< HEAD -0.4445 -0.3419 -0.3732 -0.4101 -0.4267 -0.4454 -0.7067 ======= 0.5472 0.3545 0.5042 0.5696 0.5598 0.5663 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD -0.8894 -0.8731 -0.8004 -0.8196 -0.6786 -0.7710 -0.9139 ======= 0.8169 0.7522 0.7828 0.8584 0.8602 0.8580 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD -0.8113 -0.7658 -0.6339 -0.7680 -0.6377 -0.7245 -0.8767 ======= 0.6839 0.5377 0.6043 0.7308 0.7205 0.7285 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD -0.8456 -0.8061 -0.7222 -0.7801 -0.6548 -0.7385 -0.8952 ======= 0.7254 0.6280 0.6521 0.7796 0.7731 0.7778 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD -0.7679 ======= 0.6469 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd -0.7470 <<<<<<< HEAD -0.6121 -0.7711 -0.4363 -0.6760 -0.8655 ======= 0.5708 0.7021 0.6763 0.6887 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##         EB       FIQT         BR        cl1        cl2        cl3        rep 
## -0.7241292 -0.6723208 -0.6218042 -0.6612917 -0.5691792 -0.6445125 -0.8355375

The methods are ranked according to the results above in ascending order:

##   EB FIQT   BR  cl1  cl2  cl3  rep 
##    2    3    6    4    7    5    1
=======

Fraction of significant SNPs with improved association estimates due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):

5. Bimodal distribution of effect sizes

Here we investigate the 24 different scenarios under a bimodal distribution of effect sizes. In order to create a bimodal distribution, we simulate 50% of effect sizes of the true effect SNPs from a normal distribution centered at 0 while the other half are generated from a normal distribution with mean 2.5. As above, we first have a look at the expected number of significant SNPs and the expected proportion of those in which their association estimate is significantly exaggerated.

Running the code provided in nsig_prop_bias_100sim.R, we obtain the following results:

<<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD <<<<<<< HEAD
Scenario n_samples h2 prop_effect S n_sig 5e-8 prop_bias 5e-8 prop_x 5e-8 mse 5e-8 sd(n_sig) 5e-8 sd(prop_bias) 5e-8 sd(prop_x) 5e-8 sd(mse) 5e-8
1 30,000 0.3 0.010 -1 <<<<<<< HEAD 0.61 1.0000 0.9583 0.001874 0.803 0.0000 0.1729 0.003349 ======= -0.001857 -0.001860 -0.001746 -0.001620 -0.001363 -0.001535 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 855.06 <<<<<<< HEAD 0.7741 ======= -0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.0910 0.000016 ======= -0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 19.640 0.0127 0.0090 0.000001 ======= -0.000017 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD 24.74 0.9968 0.4860 0.000474 5.181 0.0111 0.1190 0.000207 ======= -0.000903 -0.000888 -0.000878 -0.000809 -0.000780 -0.000828 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD 2820.65 ======= -0.000009 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.6243 <<<<<<< HEAD 0.0470 0.000014 ======= -0.000005 -0.000003 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 27.505 0.0085 0.0043 0.000001 ======= -0.000006 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD 85.88 0.7709 0.0882 0.000161 5.689 0.0503 0.0294 0.000041 ======= -0.001190 -0.001124 -0.001053 -0.001131 -0.000955 -0.001076 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD 574.45 ======= -0.000076 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.5513 <<<<<<< HEAD 0.0337 0.000015 12.756 0.0207 0.0080 0.000002 ======= -0.000057 -0.000073 -0.000061 -0.000069 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD 281.92 0.6252 0.0475 0.000136 8.026 0.0287 0.0118 0.000022 ======= -0.000921 -0.000843 -0.000760 -0.000888 -0.000747 -0.000844 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 729.09 <<<<<<< HEAD 0.5264 0.0288 0.000015 12.631 0.0177 0.0056 0.000001 ======= -0.000062 -0.000050 -0.000065 -0.000045 -0.000059 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD 0.44 1.0000 1.0000 0.002490 0.686 0.0000 0.0000 0.006906 ======= -0.001743 -0.001749 -0.001620 -0.001533 -0.001274 -0.001444 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 <<<<<<< HEAD 906.91 0.7750 0.0855 ======= -0.000025 -0.000021 -0.000022 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.000012 18.945 0.0132 0.0094 0.000001 ======= -0.000024 -0.000024 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD 22.50 1.0000 0.5724 0.000392 4.520 0.0000 0.1058 0.000087 ======= -0.000863 -0.000864 -0.000824 -0.000808 -0.000722 -0.000793 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD 2867.23 ======= -0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.6130 0.0442 0.000010 29.528 ======= -0.000008 -0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.0090 0.0038 0.000000
13 30,000 0.3 0.001 0 <<<<<<< HEAD 91.79 0.7793 ======= -0.001328 -0.001298 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.0867 0.000120 6.703 0.0456 0.0286 0.000026 ======= -0.001221 -0.001023 -0.001156 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD 543.36 0.5426 0.0315 0.000012 10.686 0.0230 0.0083 0.000001 ======= -0.000076 -0.000071 -0.000059 -0.000074 -0.000062 -0.000070 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD 285.41 0.6093 0.0420 0.000101 9.313 0.0282 0.0111 0.000013 ======= -0.000974 -0.000925 -0.000830 -0.000915 -0.000772 -0.000869 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 693.12 <<<<<<< HEAD 0.5289 0.0307 ======= -0.000063 -0.000051 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.000013 12.186 0.0176 0.0075 0.000001 ======= -0.000045 -0.000061 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD 0.52 1.0000 1.0000 0.001402 0.717 0.0000 0.0000 0.001109 ======= -0.001642 -0.001648 -0.001527 -0.001435 -0.001202 -0.001355 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 <<<<<<< HEAD 918.79 0.8193 0.0945 0.000012 19.846 ======= -0.000026 -0.000024 -0.000023 -0.000025 -0.000025 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.0133 0.0096 0.000001
19 30,000 0.8 0.010 1 <<<<<<< HEAD 15.89 1.0000 0.7521 0.000474 3.408 0.0000 0.1188 0.000097 ======= -0.000803 -0.000808 -0.000767 -0.000763 -0.000676 -0.000745 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 3130.86 <<<<<<< HEAD 0.6014 ======= -0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.0392 0.000010 26.113 0.0098 0.0034 0.000000 ======= -0.000013 -0.000014 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD 91.90 0.8221 0.0879 0.000119 5.901 0.0404 0.0293 0.000023 ======= -0.001310 -0.001285 -0.001181 -0.001187 -0.000990 -0.001120 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD 521.36 0.5350 0.0309 0.000012 8.481 0.0212 0.0081 0.000001 ======= -0.000088 -0.000082 -0.000068 -0.000083 -0.000069 -0.000078 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD 313.57 0.5995 0.0392 0.000095 8.843 0.0282 0.0123 0.000010 ======= -0.001040 -0.000990 -0.000882 -0.000970 -0.000808 -0.000915 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD 628.44 0.5231 ======= -0.000071 -0.000069 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.0298 0.000013 10.003 0.0184 0.0065 0.000001 ======= -0.000071 -0.000039 -0.000062 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

Next, we repeat the process illustrated in Section 2 using the third bias evaluation metric, rel_mse with a significance threshold of \(5 \times 10^{-8}\).

Summary of results for rel_mse contained in bimod_5e-8_10sim.csv:

=======

Change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in norm_5e-4_10sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 <<<<<<< HEAD -0.8652 -0.8611 -0.8801 -0.6651 -0.7987 -0.7563 -0.8911 ======= -0.9547 -0.9564 -0.8976 -0.8976 -0.8976 -0.8976 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 <<<<<<< HEAD -0.4005 -0.0961 -0.3535 2.3903 0.9330 1.5280 -0.3260 ======= -0.4092 -0.2321 -0.2925 -0.2925 -0.2925 -0.2925 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD -0.8398 ======= -0.7544 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd -0.7968 <<<<<<< HEAD -0.8554 0.4373 -0.6085 -0.2000 -0.7343 ======= -0.7341 -0.7341 -0.7341 -0.7341 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD -0.1200 0.2525 -0.0535 2.2781 1.2585 1.6702 -0.0908 ======= -0.2598 -0.0230 -0.1482 -0.1482 -0.1482 -0.1482 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD -0.1251 0.4392 0.0816 2.4621 0.9733 1.5853 -0.3879 ======= -0.8144 -0.7689 -0.7209 -0.7209 -0.7209 -0.7209 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD 0.0368 0.4734 0.7842 1.2027 0.7414 0.9233 0.0476 ======= -0.7622 -0.6931 -0.5692 -0.5692 -0.5692 -0.5692 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD -0.0272 0.9478 0.8110 2.5743 1.4722 1.9121 -0.0455 ======= -0.7694 -0.7048 -0.6354 -0.6354 -0.6354 -0.6354 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 <<<<<<< HEAD 0.0758 0.2279 0.5819 0.6036 0.7481 0.5433 0.0105 ======= -0.7383 -0.7019 -0.5638 -0.5638 -0.5638 -0.5638 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD -0.8787 -0.9023 -0.8513 -0.5169 -0.9215 -0.7961 -0.8889 ======= -0.9592 -0.9628 -0.8915 -0.8915 -0.8915 -0.8915 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 <<<<<<< HEAD -0.4038 -0.0690 -0.3713 2.4564 1.0098 1.5992 -0.3064 ======= -0.5201 -0.4514 -0.4635 -0.4635 -0.4635 -0.4635 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD -0.8078 -0.8759 -0.8852 0.0434 -0.7177 -0.4219 -0.7881 ======= -0.7985 -0.8002 -0.7631 -0.7631 -0.7631 -0.7631 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD -0.1118 0.2409 -0.1032 2.0764 1.1539 1.5229 -0.1016 ======= -0.3701 -0.2461 -0.3101 -0.3101 -0.3101 -0.3101 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD -0.3081 0.6181 0.0185 3.1434 1.3302 2.0801 -0.2942 ======= -0.8776 -0.8574 -0.7895 -0.7895 -0.7895 -0.7895 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD 0.0274 0.3868 0.5348 1.1949 0.7106 0.9010 0.0196 ======= -0.7619 -0.7069 -0.5868 -0.5868 -0.5868 -0.5868 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD -0.0590 0.6341 0.4830 1.8248 0.9890 1.3211 -0.1607 ======= -0.8161 -0.7741 -0.6944 -0.6944 -0.6944 -0.6944 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 <<<<<<< HEAD 0.0406 0.3382 0.6162 1.1115 0.9371 0.9084 0.0302 ======= -0.7202 -0.6922 -0.5646 -0.5646 -0.5646 -0.5646 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD -0.9393 -0.9454 -0.9415 -0.7304 -0.9411 -0.9054 -0.8765 ======= -0.9499 -0.9537 -0.8838 -0.8838 -0.8838 -0.8838 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 <<<<<<< HEAD -0.4866 -0.2098 -0.4695 2.7316 0.9980 1.7135 -0.3306 ======= -0.5479 -0.4947 -0.4935 -0.4935 -0.4935 -0.4935 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD -0.8511 -0.9174 -0.9161 -0.2153 -0.7834 -0.5711 -0.8449 ======= -0.7875 -0.7914 -0.7520 -0.7520 -0.7520 -0.7520 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 <<<<<<< HEAD -0.1137 0.3166 -0.1154 2.1944 1.2935 1.6506 -0.0561 ======= -0.4365 -0.3315 -0.3660 -0.3660 -0.3660 -0.3660 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD -0.5084 0.0929 -0.3487 2.3442 0.7355 1.3987 -0.3590 ======= -0.8878 -0.8709 -0.8010 -0.8010 -0.8010 -0.8010 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD 0.0575 0.3490 0.5646 1.2555 0.9525 0.9916 -0.0665 ======= -0.8183 -0.7712 -0.6358 -0.6358 -0.6358 -0.6358 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD 0.0243 0.8020 0.4923 2.3288 1.3454 1.7375 -0.0716 ======= -0.8484 -0.8077 -0.7194 -0.7194 -0.7194 -0.7194 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD 0.0481 0.2526 0.4966 0.6978 0.5453 0.5735 0.0543 ======= -0.7536 -0.7355 -0.6041 -0.6041 -0.6041 -0.6041 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##          EB        FIQT          BR         cl1         cl2         cl3 
## -0.31398333  0.02909167 -0.07000000  1.38432500  0.55651667  0.87122917 
##         rep 
## -0.31077083

The methods are ranked according to the results above in ascending order:

##   EB FIQT   BR  cl1  cl2  cl3  rep 
##    1    4    3    7    5    6    2

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:

\(~\) \(~\) \(~\) \(~\)

6. Evaluating methods which use both replication and discovery summary statistics

In this section, we proceed to comparing Winner’s Curse adjustment methods which use summary statistics from both the discovery and a replication GWAS. We first consider the situation in which the replication and discovery GWASs are of the same size. As an interesting comparison, we have also included the empirical Bayes method here which uses information from just the discovery GWAS. The significance threshold is set to 5e-8 and in this instance, the effect distribution is normal.

Summary of results for rel_mse contained in replicate_norm_5e-8_10sim.csv:

=======

Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-4}\):

4. Bimodal distribution of effect sizes

Here we investigate the 24 different scenarios under a bimodal distribution of effect sizes. In order to create a bimodal distribution, we simulate 50% of effect sizes of the true effect SNPs from a normal distribution centered at 0 while the other half are generated from a normal distribution with mean 2.5. As above, we first have a look at the expected number of significant SNPs and the expected proportion of those in which their association estimate is exaggerated.

Running the code provided in nsig_prop_bias_100sim.R, we obtain the following results:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min MSE_min_sp
1 30,000 0.3 0.010 -1 -0.6882 -0.9846 -0.7103 -0.7103 -0.9982 -0.9833 -0.9470 -0.9532
2 300,000 0.3 0.010 -1 -0.3039 -0.2882 -0.2875 -0.5715 -0.5010 -0.5468 -0.4586 -0.5775
3 30,000 0.8 0.010 -1 -0.6578 -0.7883 -0.7925 -0.6863 -0.8269 -0.8061 -0.7867 -0.7768
4 300,000 0.8 0.010 -1 -0.0822 -0.0425 -0.0432 -0.5127 -0.4389 -0.4863 -0.2933 -0.5106
5 30,000 0.3 0.001 -1 -0.1343 -0.2324 -0.2379 -0.5933 -0.4923 -0.5561 -0.4101 -0.5825
6 300,000 0.3 0.001 -1 0.0362 0.0439 0.0419 -0.4850 -0.4431 -0.4703 -0.2265 -0.4209
7 30,000 0.8 0.001 -1 0.0393 -0.0825 -0.0868 -0.5366 -0.4908 -0.5311 -0.3227 -0.4751
8 300,000 0.8 0.001 -1 0.0787 -0.0052 -0.0096 -0.4909 -0.4644 -0.4833 -0.2626 -0.4641
9 30,000 0.3 0.010 0 -0.6875 -0.6995 -0.6209 -0.5615 -0.8109 -0.7351 -0.6391 -0.6923
10 300,000 0.3 0.010 0 -0.2877 -0.3129 -0.3139 -0.5805 -0.5447 -0.5762 -0.4770 -0.5850
11 30,000 0.8 0.010 0 -0.6383 -0.7218 -0.7248 -0.6759 -0.7999 -0.7777 -0.7399 -0.7385
12 300,000 0.8 0.010 0 -0.0966 -0.1277 -0.1278 -0.5285 -0.4651 -0.5063 -0.3467 -0.5265
13 30,000 0.3 0.001 0 -0.1844 -0.2980 -0.3077 -0.5853 -0.5460 -0.5770 -0.4712 -0.5594
14 300,000 0.3 0.001 0 0.0482 0.0126 0.0126 -0.5011 -0.4424 -0.4783 -0.2647 -0.4252
15 30,000 0.8 0.001 0 0.0058 -0.0662 -0.0721 -0.5415 -0.4477 -0.5004 -0.3086 -0.4029
16 300,000 0.8 0.001 0 0.1245 -0.0364 -0.0358 -0.5072 -0.4660 -0.4906 -0.2879 -0.4689
17 30,000 0.3 0.010 1 -0.6981 -0.8833 -0.8371 -0.6669 -0.9011 -0.8732 -0.8594 -0.8607
18 300,000 0.3 0.010 1 -0.2217 -0.2487 -0.2494 -0.5630 -0.5114 -0.5477 -0.4266 -0.5653
19 30,000 0.8 0.010 1 -0.5748 -0.6226 -0.6211 -0.6454 -0.7037 -0.6968 -0.6611 -0.6794
20 300,000 0.8 0.010 1 -0.0721 -0.0740 -0.0743 -0.5209 -0.4465 -0.4912 -0.3145 -0.5166
21 30,000 0.3 0.001 1 -0.1841 -0.2900 -0.2943 -0.5556 -0.5136 -0.5407 -0.4437 -0.5233
22 300,000 0.3 0.001 1 0.0408 -0.0108 -0.0172 -0.4975 -0.4613 -0.4878 -0.2637 -0.3899
23 30,000 0.8 0.001 1 0.0218 -0.1438 -0.1439 -0.5176 -0.4641 -0.5037 -0.3492 -0.4009
24 300,000 0.8 0.001 1 0.1391 -0.0311 -0.0334 -0.5134 -0.4676 -0.4952 -0.2823 -0.4587

\(~\) \(~\) \(~\) \(~\)

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##         EB        rep     UMVCUE    cl1_com    cl2_MLE    cl3_MSE    MSE_min 
## -0.2073875 -0.2889167 -0.2744583 -0.5645167 -0.5686500 -0.5892167 -0.4517958 
## MSE_min_sp 
## -0.5647583

The methods are ranked according to the results above in ascending order:

##         EB        rep     UMVCUE    cl1_com    cl2_MLE    cl3_MSE    MSE_min 
##          8          6          7          4          2          1          5 
## MSE_min_sp 
##          3

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation with a replication and discovery GWAS of equal size, using a significance threshold of \(5 \times 10^{-8}\):

\(~\) \(~\) \(~\) \(~\)

We also look at how the methods compare when the replication data set is 50% that of the discovery data set.

Summary of results for rel_mse contained in replicate_norm_5e-8_10sim_halfrep.csv:

Scenario n_samples h2 prop_effect S EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min MSE_min_sp
1 30,000 0.3 0.010 -1 -0.9465 -0.8273 -0.7601 -0.5695 -0.8224 -0.8149 -0.8362 -0.8436
2 300,000 0.3 0.010 -1 -0.3233 0.5071 0.5072 -0.3826 -0.2016 -0.2700 0.0454 -0.3504
3 30,000 0.8 0.010 -1 -0.7190 -0.2438 -0.2946 -0.4565 -0.5059 -0.4921 -0.3910 -0.5307
4 300,000 0.8 0.010 -1 -0.0825 0.8737 0.8728 -0.3374 -0.1311 -0.2226 0.2818 -0.2589
5 30,000 0.3 0.001 -1 -0.2625 0.5368 0.5098 -0.3689 -0.1362 -0.2282 0.0930 -0.2879
6 300,000 0.3 0.001 -1 0.0100 0.8443 0.8435 -0.3590 -0.2866 -0.3273 0.2495 -0.2000
7 30,000 0.8 0.001 -1 0.0173 0.9937 0.9918 -0.3338 -0.1174 -0.2075 0.3843 -0.0774
8 300,000 0.8 0.001 -1 0.0353 0.9906 0.9703 -0.3170 -0.2633 -0.2904 0.3297 -0.1619
9 30,000 0.3 0.010 0 -0.3697 -0.8137 -0.5393 -0.4290 -0.8910 -0.8355 -0.7105 -0.7885
10 300,000 0.3 0.010 0 -0.2771 0.4872 0.4864 -0.3953 -0.2142 -0.2912 0.0418 -0.3643
11 30,000 0.8 0.010 0 -0.6187 -0.3528 -0.3591 -0.4831 -0.6165 -0.5994 -0.4701 -0.5972
12 300,000 0.8 0.010 0 -0.1012 0.7565 0.7555 -0.3575 -0.1833 -0.2643 0.1983 -0.2922
13 30,000 0.3 0.001 0 -0.2337 0.5986 0.5843 -0.4006 -0.1511 -0.2390 0.1228 -0.2357
14 300,000 0.3 0.001 0 0.0483 0.9139 0.8996 -0.3448 -0.1797 -0.2550 0.2947 -0.1101
15 30,000 0.8 0.001 0 -0.0499 0.7956 0.7857 -0.3524 -0.1500 -0.2412 0.2286 -0.0014
16 300,000 0.8 0.001 0 0.0733 1.1343 1.1239 -0.3104 -0.2131 -0.2594 0.4253 -0.1044
17 30,000 0.3 0.010 1 -0.7573 -0.7222 -0.4972 -0.4841 -0.7932 -0.7483 -0.7040 -0.7531
18 300,000 0.3 0.010 1 -0.2005 0.6098 0.6091 -0.3847 -0.2005 -0.2781 0.1110 -0.3313
19 30,000 0.8 0.010 1 -0.6205 -0.2587 -0.2740 -0.4668 -0.5206 -0.5183 -0.4250 -0.5586
20 300,000 0.8 0.010 1 -0.0677 0.8176 0.8163 -0.3474 -0.1834 -0.2612 0.2395 -0.2714
21 30,000 0.3 0.001 1 -0.1625 0.6967 0.7020 -0.3761 -0.1921 -0.2717 0.1407 -0.1467
22 300,000 0.3 0.001 1 0.0335 0.9038 0.8923 -0.3463 -0.2270 -0.2812 0.2960 -0.0828
23 30,000 0.8 0.001 1 0.0525 0.7890 0.7675 -0.3411 -0.1669 -0.2543 0.2224 0.0095
24 300,000 0.8 0.001 1 0.1719 1.0337 1.0202 -0.3224 -0.2172 -0.2656 0.3748 -0.1019

\(~\) \(~\) \(~\) \(~\)

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##          EB         rep      UMVCUE     cl1_com     cl2_MLE     cl3_MSE 
## -0.22293750  0.46101667  0.47557917 -0.38611250 -0.31517917 -0.36319583 
##     MSE_min  MSE_min_sp 
##  0.02261667 -0.31003750

The methods are ranked according to the results above in ascending order:

##         EB        rep     UMVCUE    cl1_com    cl2_MLE    cl3_MSE    MSE_min 
##          5          7          8          1          3          2          6 
## MSE_min_sp 
##          4

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation with a replication GWAS 50% of the size of the discovery GWAS, using a significance threshold of \(5 \times 10^{-8}\):

It is very interesting here how empirical Bayes behaves better in nearly all of the scenarios compared to UMVCUE, the replication method and the MSE minimization method.

\(~\) \(~\) \(~\) \(~\)

Finally, we look at how the methods compare when the replication data set is 10% that of the discovery data set.

Summary of results for rel_mse contained in replicate_norm_5e-8_10sim_10pc.csv:

Scenario n_samples h2 prop_effect S EB rep UMVCUE cl1_com cl2_MLE cl3_MSE MSE_min MSE_min_sp
1 30,000 0.3 0.010 -1 -0.9576 -0.4613 -0.1714 -0.1714 -0.8319 -0.8711 -0.5592 -0.6053
2 300,000 0.3 0.010 -1 -0.3287 6.3266 6.3019 -0.1162 0.8340 0.6565 3.4858 1.1209
3 30,000 0.8 0.010 -1 -0.6830 2.4395 2.2218 -0.1057 -0.0746 -0.0838 1.2711 0.2963
4 300,000 0.8 0.010 -1 -0.0847 8.4091 8.4020 -0.0973 0.7255 0.5296 4.7008 1.6199
5 30,000 0.3 0.001 -1 -0.2085 6.0848 5.8081 -0.1262 0.9333 0.7615 3.3364 1.1594
6 300,000 0.3 0.001 -1 0.0257 9.2150 9.0789 -0.0912 0.2946 0.1921 5.2340 2.0676
7 30,000 0.8 0.001 -1 -0.0166 8.4894 8.3256 -0.1246 0.8706 0.6712 4.7567 1.8498
8 300,000 0.8 0.001 -1 0.0648 8.8348 8.5369 -0.0897 0.1484 0.0826 4.9340 1.8612
9 30,000 0.3 0.010 0 -0.7862 1.4177 0.7516 -0.0651 -0.4803 -0.4362 0.8726 0.8886
10 300,000 0.3 0.010 0 -0.2930 6.3992 6.3831 -0.1067 0.7452 0.5794 3.5436 1.1743
11 30,000 0.8 0.010 0 -0.6628 2.2452 2.0602 -0.1431 -0.0400 -0.0622 1.1103 0.1811
12 300,000 0.8 0.010 0 -0.1000 8.0019 7.9933 -0.0949 0.7097 0.5303 4.4779 1.5427
13 30,000 0.3 0.001 0 -0.1738 6.2345 5.9534 -0.1046 0.8201 0.6574 3.4786 1.4050
14 300,000 0.3 0.001 0 0.0177 9.0005 8.8139 -0.0930 0.4598 0.3378 5.1128 2.1003
15 30,000 0.8 0.001 0 -0.0496 7.7050 7.5064 -0.1070 0.5751 0.4182 4.3114 2.0258
16 300,000 0.8 0.001 0 0.1019 9.1618 8.9700 -0.0813 0.1711 0.1034 5.1288 1.9984
17 30,000 0.3 0.010 1 -0.7917 0.7464 0.8698 -0.1488 -0.3068 -0.3178 0.2556 0.0458
18 300,000 0.3 0.010 1 -0.2123 6.5887 6.5606 -0.1087 0.7792 0.6088 3.6541 1.2120
19 30,000 0.8 0.010 1 -0.6125 3.1323 3.1115 -0.1485 0.0304 -0.0062 1.7162 0.3105
20 300,000 0.8 0.010 1 -0.0764 8.0191 7.9946 -0.0993 0.6163 0.4489 4.4762 1.5375
21 30,000 0.3 0.001 1 -0.0493 6.7867 6.5866 -0.1167 0.9210 0.7058 3.7077 1.6689
22 300,000 0.3 0.001 1 0.0415 8.8260 8.5701 -0.0936 0.3972 0.2748 4.9647 2.1855
23 30,000 0.8 0.001 1 0.0125 8.4717 8.2306 -0.0925 0.6368 0.4645 4.7694 2.4695
24 300,000 0.8 0.001 1 0.1759 9.1156 8.8594 -0.0802 0.2757 0.1787 5.1389 2.0793

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##         EB        rep     UMVCUE    cl1_com    cl2_MLE    cl3_MSE    MSE_min 
## -0.2352792  6.2995917  6.1549542 -0.1085958  0.3837667  0.2676750  3.4949333 
## MSE_min_sp 
##  1.3414583

The methods are ranked according to the results above in ascending order:

##         EB        rep     UMVCUE    cl1_com    cl2_MLE    cl3_MSE    MSE_min 
##          1          8          7          2          4          3          6 
## MSE_min_sp 
##          5

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation with a replication GWAS 10% of the size of the discovery GWAS, using a significance threshold of \(5 \times 10^{-8}\):

In this situation, the empirical Bayes method tends to behave better than all of the other methods.

\(~\) \(~\) \(~\) \(~\)

7. Winner’s Curse and Binary Traits

Under the same 24 scenarios, we have a look at simulating summary statistics for a binary trait and the corresponding performance of the various Winner’s Curse adjustment methods. Below is a summary of the results contained in binary_norm_nsig_prop_bias_5e-8_10sim.csv and binary_norm_nsig_prop_bias_5e-4_10sim.csv. Compared to the quantitative trait above, we witness a lot less SNPs meeting the significance threshold of 5e-8 under many of the scenarios.

=======

Next, we repeat the process illustrated in Section 2 using the same bias evaluation metrics with a significance threshold of \(5 \times 10^{-8}\).

Summary of results for flb contained in bimod_5e-8_10sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S n_sig 5e-8 prop_bias 5e-8 prop_x 5e-8 mse 5e-8 n_sig 5e-4 prop_bias 5e-4 prop_x 5e-4 mse 5e-4 sd(n_sig) 5e-8 sd(prop_bias) 5e-8 sd(prop_x) 5e-8 sd(mse) 5e-8 sd(n_sig) 5e-4 sd(prop_bias) 5e-4 sd(prop_x) 5e-4 sd(mse) 5e-4
1 30,000 0.3 0.010 -1 0.1 1.0000 1.0000 0.043205 503.2 1.0000 1.0000 0.024306 0.316 NA NA NA 19.384 0.0000 0.0000 0.001972
2 300,000 0.3 0.010 -1 <<<<<<< HEAD 0.4 1.0000 1.0000 0.001337 563.2 1.0000 0.9590 0.002258 0.516 0.0000 0.0000 0.000347 23.748 0.0000 0.0107 0.000134 ======= 0.6451 0.4586 0.6244 0.5311 0.4978 0.5123 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD 2.3 1.0000 0.6667 0.018844 687.4 0.9960 0.8229 0.019493 1.337 0.0000 0.2976 0.009237 17.658 0.0028 0.0173 0.001161 ======= 0.9403 0.8540 0.9229 0.6343 0.8282 0.7446 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD 1353.9 0.7098 0.0732 0.000215 3870.3 0.7067 0.1751 0.000481 16.749 0.0106 0.0058 0.000011 44.672 0.0067 0.0044 0.000016 ======= 0.5674 0.2793 0.5434 0.5159 0.4857 0.4994 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD 0.2 1.0000 1.0000 0.023959 510.7 1.0000 0.9934 0.024185 0.422 0.0000 0.0000 0.011519 19.568 0.0000 0.0029 0.000956 ======= 0.5763 0.3771 0.5537 0.5583 0.5241 0.5396 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD 51.5 0.7965 0.1205 0.000298 713.3 0.9227 0.7224 0.001781 5.339 0.0565 0.0552 0.000110 28.960 0.0073 0.0099 0.000106 ======= 0.4930 0.1344 0.2900 0.5079 0.4801 0.5009 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD 130.2 0.7189 0.0761 0.002359 834.1 0.8651 0.6177 0.015121 7.657 0.0337 0.0238 0.000643 28.097 0.0080 0.0168 0.000689 ======= 0.5241 0.2150 0.3208 0.5136 0.4816 0.4960 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 <<<<<<< HEAD 626.3 0.5347 0.0336 0.000186 1251.8 0.7170 0.4159 0.001057 17.982 0.0182 0.0052 0.000015 30.025 0.0117 0.0109 0.000059 ======= 0.4778 0.1009 0.2828 0.4994 0.4718 0.4882 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 0.1 1.0000 1.0000 0.031993 498.8 1.0000 1.0000 0.024378 0.316 NA NA NA 16.518 0.0000 0.0000 0.000919
10 300,000 0.3 0.010 0 <<<<<<< HEAD 0.3 1.0000 1.0000 0.001994 580.4 0.9998 0.9448 0.002149 0.483 0.0000 0.0000 0.002061 31.178 0.0005 0.0120 0.000140 ======= 0.6546 0.4648 0.6411 0.5354 0.4981 0.5120 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD 5.3 0.9833 0.4772 0.003954 718.3 0.9925 0.7883 0.017537 2.111 0.0527 0.2430 0.001587 25.755 0.0032 0.0111 0.000823 ======= 0.9465 0.8919 0.9176 0.6403 0.8035 0.7169 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD 1338.0 0.6829 0.0664 0.000120 3678.1 0.7050 0.1873 0.000463 22.376 0.0124 0.0047 0.000006 20.739 0.0059 0.0066 0.000024 ======= 0.5517 0.2668 0.5375 0.5100 0.4817 0.4939 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD 0.0 NA NA NA 514.1 1.0000 0.9941 0.023966 0.000 NA NA NA 11.930 0.0000 0.0033 0.000874 ======= 0.6741 0.3418 0.5541 0.5553 0.5044 0.5269 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD 56.4 0.7825 0.1075 0.000141 703.9 0.9209 0.7296 0.001785 5.661 0.0628 0.0393 0.000030 24.016 0.0068 0.0113 0.000134 ======= 0.4773 0.1266 0.2885 0.5025 0.4715 0.4954 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD 130.3 0.6866 0.0622 0.001186 816.3 0.8683 0.6312 0.015436 8.111 0.0497 0.0275 0.000160 16.180 0.0073 0.0160 0.000864 ======= 0.5144 0.1954 0.3006 0.5071 0.4853 0.4959 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 <<<<<<< HEAD 585.6 0.5541 0.0330 0.000133 1218.8 0.7286 0.4246 0.001058 15.785 0.0179 0.0082 0.000013 20.115 0.0112 0.0155 0.000067 ======= 0.4806 0.1032 0.3029 0.5099 0.4783 0.4992 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 0.1 1.0000 1.0000 0.029079 505.2 1.0000 1.0000 0.024162 0.316 NA NA NA 15.259 0.0000 0.0000 0.001306
18 300,000 0.3 0.010 1 <<<<<<< HEAD 0.6 1.0000 0.8750 0.001202 580.7 0.9991 0.9277 0.002092 0.843 0.0000 0.2500 0.000601 20.907 0.0012 0.0087 0.000144 ======= 0.6937 0.5278 0.6818 0.5512 0.5114 0.5289 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD 8.0 0.9546 0.3713 0.003110 730.7 0.9851 0.7653 0.017206 2.582 0.0761 0.2686 0.001241 26.779 0.0045 0.0202 0.001488 ======= 0.9838 0.9449 0.9713 0.6567 0.9150 0.7789 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 <<<<<<< HEAD 1343.7 0.6683 0.0574 0.000101 3445.6 0.6977 0.1899 0.000435 26.030 0.0139 0.0062 0.000003 42.717 0.0110 0.0075 0.000024 ======= 0.5555 0.2588 0.5421 0.5161 0.4858 0.4998 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD 0.1 1.0000 1.0000 0.028158 502.1 0.9998 0.9924 0.023958 0.316 NA NA NA 22.951 0.0006 0.0037 0.001363 ======= 0.6794 0.4069 0.6073 0.5699 0.5357 0.5488 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD 64.5 0.7587 0.0862 0.000115 697.8 0.9203 0.7345 0.001734 6.654 0.0399 0.0416 0.000032 28.142 0.0122 0.0132 0.000078 ======= 0.4896 0.1133 0.2642 0.5107 0.4895 0.5057 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD 134.2 0.6752 0.0730 0.001053 793.5 0.8737 0.6537 0.016008 5.903 0.0244 0.0277 0.000162 21.629 0.0085 0.0167 0.000750 ======= 0.4903 0.1855 0.3245 0.5156 0.4776 0.4956 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD 532.7 0.5474 0.0316 0.000106 1159.1 0.7369 0.4396 0.001107 12.763 0.0103 0.0064 0.000010 18.806 0.0080 0.0085 0.000067 ======= 0.4819 0.0918 0.2496 0.5054 0.4851 0.4986 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

We plot \(z\) vs \(\text{bias}\) for the 24 different scenarios with a simulated binary trait. Similar to above, the points corresponding to SNPs which are significantly biased and are significant at a threshold of \(5 \times 10^{-4}\) are coloured in navy .

\(~\) \(~\) \(~\) \(~\)

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in norm_5e-8_10sim.csv:

=======

Fraction of significant SNPs with improved association estimates due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:

\(~\) \(~\) \(~\) \(~\)

Summary of results for mse contained in bimod_5e-8_10sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 <<<<<<< HEAD -0.8321 -0.7771 -0.9121 -0.6689 -0.6791 -0.6746 -0.9954 ======= -0.001688 -0.001791 -0.001807 -0.001527 -0.001645 -0.001620 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 -0.9371 -0.9499 <<<<<<< HEAD -0.8904 -0.7777 ======= -0.000006 0.000038 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd -0.9229 <<<<<<< HEAD -0.8986 -0.9533 ======= 0.000024 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD -0.7689 -0.7803 -0.7013 -0.0147 -0.7514 -0.4770 -0.8230 ======= -0.000412 -0.000448 -0.000455 0.000056 -0.000353 -0.000196 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 -0.2148 <<<<<<< HEAD 0.1763 ======= 0.000004 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd -0.0974 <<<<<<< HEAD 2.4029 1.1847 ======= 0.000029 0.000017 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 1.6734 -0.2074
5 30,000 0.3 0.001 -1 <<<<<<< HEAD -0.8933 -0.8504 -0.9085 -0.9945 -0.8166 -0.9368 -0.9954 ======= -0.000022 0.000036 -0.000020 0.000376 0.000127 0.000229 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 -0.2884 0.6545 0.4088 2.4080 <<<<<<< HEAD 0.9838 1.5630 -0.3425 ======= 0.000011 0.000013 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD -0.1623 1.0327 0.6538 3.7188 1.7049 2.5403 0.0034 ======= -0.000010 0.000087 0.000078 0.000272 0.000144 0.000194 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 0.0580 <<<<<<< HEAD 0.3212 ======= 0.000004 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.6455 <<<<<<< HEAD 0.7320 0.6382 0.6034 -0.0108 ======= 0.000012 0.000013 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD -0.8812 -0.8610 -0.9138 -0.9951 -0.8308 -0.9421 -0.9792 ======= -0.000764 -0.000765 -0.000752 -0.000490 -0.000768 -0.000699 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 -0.9555 -0.9388 -0.9438 <<<<<<< HEAD -0.7604 -0.8773 ======= 0.000029 0.000011 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd -0.8638 -0.8946
11 30,000 0.8 0.010 0 <<<<<<< HEAD -0.6630 -0.6213 -0.4645 0.6283 -0.5625 -0.0915 -0.7332 ======= -0.000332 -0.000359 -0.000361 -0.000005 -0.000280 -0.000172 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 -0.2002 <<<<<<< HEAD 0.1320 ======= 0.000003 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd -0.0877 2.1744 1.0360 <<<<<<< HEAD 1.4948 -0.2334 ======= 0.000015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD NA ======= -0.000035 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd NA <<<<<<< HEAD NA NA NA NA NA ======= -0.000009 0.000280 0.000122 0.000187 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 -0.2525 0.8169 0.5802 2.9517 <<<<<<< HEAD 1.2249 1.9364 -0.2569 ======= 0.000008 0.000010 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD -0.1714 0.6272 0.4434 2.2274 1.1211 1.5730 -0.2541 ======= -0.000003 0.000067 0.000053 0.000218 0.000120 0.000159 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 0.0511 0.4392 <<<<<<< HEAD 0.7611 1.2359 0.7964 0.9527 0.0005 ======= 0.000008 0.000014 0.000013 0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD -0.8904 -0.8600 -0.9145 -0.9951 -0.8296 -0.9416 -0.9994 ======= -0.001252 -0.001142 -0.001296 -0.000599 -0.000869 -0.000742 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 -0.6472 -0.7991 -0.4708 -0.0551 -0.8710 -0.5756 <<<<<<< HEAD -0.8783 ======= 0.000018 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD -0.7983 -0.7001 -0.6627 0.2694 -0.5722 -0.2399 -0.7207 ======= -0.000374 -0.000397 -0.000397 -0.000054 -0.000337 -0.000228 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 -0.1584 0.1997 -0.0140 <<<<<<< HEAD 2.1550 ======= 0.000020 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 1.0719 <<<<<<< HEAD 1.5083 -0.1636 ======= 0.000015 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD -0.8899 -0.8676 -0.9156 -0.9954 -0.8391 -0.9451 -0.9961 ======= -0.000036 0.000021 -0.000038 0.000289 0.000098 0.000177 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 -0.1822 <<<<<<< HEAD 0.8427 0.6334 2.7302 1.2514 1.8581 -0.3071 ======= 0.000004 0.000006 0.000010 0.000006 0.000007 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD -0.0358 0.7604 0.5807 2.4714 1.1717 1.6978 -0.0967 ======= 0.000009 0.000072 0.000042 0.000193 0.000168 0.000158 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 0.0415 <<<<<<< HEAD 0.3314 ======= 0.000004 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.6490 <<<<<<< HEAD 0.9037 ======= 0.000012 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd 0.7276 <<<<<<< HEAD 0.7336 -0.0583 ======= 0.000009 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##         EB       FIQT         BR        cl1        cl2        cl3        rep 
## -0.4640130 -0.1161478 -0.1539652  0.9022696  0.1895696  0.4586174 -0.5171957

The methods are ranked according to the results above in ascending order:

##   EB FIQT   BR  cl1  cl2  cl3  rep 
##    2    4    3    7    5    6    1

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation for a binary trait, using a significance threshold of \(5 \times 10^{-8}\):

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in norm_5e-4_10sim.csv:

=======

Change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:

\(~\) \(~\) \(~\) \(~\)

Summary of results for rel_mse contained in bimod_5e-8_10sim.csv:

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
Scenario n_samples h2 prop_effect S EB FIQT BR cl1 cl2 cl3 rep
1 30,000 0.3 0.010 -1 <<<<<<< HEAD -0.9941 -0.9988 -0.9147 -0.8562 -0.6989 -0.7985 -0.9300 ======= -0.8882 -0.9082 -0.9140 -0.7189 -0.8594 -0.8248 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
2 300,000 0.3 0.010 -1 <<<<<<< HEAD -0.9753 -0.9777 -0.9085 -0.8461 -0.7023 -0.7959 -0.9218 ======= -0.3975 -0.0841 -0.3577 2.4220 0.9430 1.5479 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
3 30,000 0.8 0.010 -1 <<<<<<< HEAD -0.9080 -0.9117 -0.8659 -0.8112 -0.6997 -0.7788 -0.9099 ======= -0.7891 -0.8544 -0.8707 0.1311 -0.6646 -0.3553 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
4 300,000 0.8 0.010 -1 <<<<<<< HEAD -0.3577 -0.1615 -0.2376 -0.1789 -0.3066 -0.2761 -0.6454 ======= -0.1208 0.2653 -0.0497 2.1411 1.2413 1.5983 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
5 30,000 0.3 0.001 -1 <<<<<<< HEAD -0.9907 -0.9943 -0.9130 -0.8427 -0.6935 -0.7882 -0.9280 ======= -0.1081 0.3147 -0.0549 2.3712 0.8515 1.4679 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
6 300,000 0.3 0.001 -1 <<<<<<< HEAD -0.8308 -0.8050 -0.7557 -0.7759 -0.6612 -0.7417 -0.9043 ======= 0.0300 0.4174 0.6944 1.2012 0.7558 0.9311 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
7 30,000 0.8 0.001 -1 <<<<<<< HEAD -0.8103 -0.7525 -0.7035 -0.7686 -0.6481 -0.7314 -0.8896 ======= -0.0637 0.6496 0.5894 1.9649 1.0608 1.4161 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
8 300,000 0.8 0.001 -1 <<<<<<< HEAD -0.7437 -0.6824 -0.5465 -0.7248 -0.5879 -0.6805 -0.8323 ======= 0.0520 0.2872 0.6181 0.7796 0.8747 0.7034 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
9 30,000 0.3 0.010 0 <<<<<<< HEAD -0.9954 -0.9987 -0.9134 -0.8510 -0.6963 -0.7942 -0.9299 ======= -0.9917 -0.9925 -0.9766 -0.6280 -0.9969 -0.9042 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
10 300,000 0.3 0.010 0 <<<<<<< HEAD -0.9755 -0.9775 -0.9024 -0.8420 -0.6984 -0.7915 -0.9275 ======= -0.3946 -0.1075 -0.3807 2.4194 0.9448 1.5473 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
11 30,000 0.8 0.010 0 <<<<<<< HEAD -0.9191 -0.9307 -0.8659 -0.8246 -0.6966 -0.7831 -0.9154 ======= -0.7912 -0.8434 -0.8545 0.0612 -0.6428 -0.3631 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
12 300,000 0.8 0.010 0 <<<<<<< HEAD -0.4626 -0.3698 -0.3970 -0.3819 -0.4431 -0.4437 -0.7320 ======= -0.1084 0.2665 -0.0957 2.0651 1.1895 1.5358 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
13 30,000 0.3 0.001 0 <<<<<<< HEAD -0.9932 -0.9965 -0.9120 -0.8503 -0.6966 -0.7943 -0.9299 ======= -0.3183 0.5595 -0.0743 2.5959 1.1429 1.7360 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
14 300,000 0.3 0.001 0 <<<<<<< HEAD -0.8928 -0.8802 -0.8046 -0.8170 -0.6802 -0.7708 -0.9158 ======= 0.0285 0.3988 0.5762 1.1694 0.7158 0.8962 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
15 30,000 0.8 0.001 0 <<<<<<< HEAD -0.8400 -0.8168 -0.7571 -0.7792 -0.6582 -0.7405 -0.9031 ======= -0.0257 0.6190 0.4922 1.9898 1.0994 1.4558 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
16 300,000 0.8 0.001 0 <<<<<<< HEAD -0.7472 -0.6972 -0.5741 -0.7391 -0.6133 -0.6978 -0.8599 ======= 0.0493 0.3351 0.6395 1.1097 1.0027 0.9217 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
17 30,000 0.3 0.010 1 <<<<<<< HEAD -0.9962 -0.9989 -0.9136 -0.8505 -0.6966 -0.7940 -0.9294 ======= -0.9631 -0.8789 -0.9976 -0.4609 -0.6684 -0.5710 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
18 300,000 0.3 0.010 1 <<<<<<< HEAD -0.9709 -0.9739 -0.9010 -0.8378 -0.6965 -0.7881 -0.9234 ======= -0.4894 -0.2587 -0.4803 2.4502 0.8630 1.5173 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
19 30,000 0.8 0.010 1 <<<<<<< HEAD -0.9072 -0.9181 -0.8523 -0.8179 -0.6914 -0.7770 -0.9155 ======= -0.8711 -0.9199 -0.9222 -0.0954 -0.7784 -0.5168 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
20 300,000 0.8 0.010 1 <<<<<<< HEAD -0.5086 -0.4340 -0.4411 -0.4843 -0.4899 -0.5149 -0.7707 ======= -0.1136 0.3018 -0.1147 2.1294 1.2529 1.5992 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
21 30,000 0.3 0.001 1 <<<<<<< HEAD -0.9913 -0.9961 -0.9128 -0.8582 -0.7000 -0.8001 -0.9274 ======= -0.3030 0.1919 -0.2896 2.3145 0.7983 1.4281 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
22 300,000 0.3 0.001 1 <<<<<<< HEAD -0.8968 -0.8857 -0.8094 -0.8129 -0.6767 -0.7661 -0.9155 ======= 0.0525 0.3181 0.4888 0.7855 0.5018 0.6078 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
23 30,000 0.8 0.001 1 <<<<<<< HEAD -0.8700 -0.8444 -0.7759 -0.7960 -0.6664 -0.7523 -0.9048 ======= 0.0926 0.7189 0.4250 1.9248 1.6475 1.5694 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd
24 300,000 0.8 0.001 1 <<<<<<< HEAD -0.7955 -0.7539 -0.6205 -0.7730 -0.6339 -0.7268 -0.8704 ======= 0.0418 0.3081 0.5540 0.8720 0.5590 0.6728 >>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd

\(~\) \(~\) \(~\) \(~\)

<<<<<<< HEAD

The mean relative change in average MSE over all significant SNPs across the 24 scenarios is obtained for each method:

##         EB       FIQT         BR        cl1        cl2        cl3        rep 
## -0.8488708 -0.8231792 -0.7582708 -0.7550042 -0.6430125 -0.7219292 -0.8846625

The methods are ranked according to the results above in ascending order:

##   EB FIQT   BR  cl1  cl2  cl3  rep 
##    2    3    4    5    7    6    1

\(~\) \(~\) \(~\) \(~\)

Relative change in average MSE over all significant SNPs due to method implementation for a binary trait, using a significance threshold of \(5 \times 10^{-4}\):

=======

Relative change in average MSE over all significant SNPs due to method implementation, using a significance threshold of \(5 \times 10^{-8}\) and a bimodal distribution of effect sizes:

\(~\) \(~\) \(~\) \(~\)

>>>>>>> e9faac94aafb297e931ca9723f5e497f49b825bd